Supporting self-regulated-learning in colonoscopy training-A comparison cohort trial.

Aim: This study aimed to demonstrate that using a self-regulated learning (SRL) approach can improve colonoscopy performance skills. Background: Colonoscopy is the gold standard for detecting colorectal cancer and removing its precursors: polyps. Acquiring proficiency in colonoscopy is challenging, requiring completion of several hundred procedures. SRL seems to be beneficial to help trainees acquire competencies in regulating their future learning processes and enhance the outcomes of current learning situations. SRL is a learner-centred approach that refers to a trainee's ability to understand and control their learning environment, including cognitions, motivations and emotions. The key abilities include self- and situational awareness, task analysis, and strategic planning. This study is the first to use an SRL approach for workplace-based colonoscopy training. Methods: In this comparison cohort trial, participants used two SRL supports: a self-review of videotaped performance, and an online learning platform with procedural and conceptual knowledge about colonoscopy. In the control cohort, participants performed patient-based colonoscopy as usual in their departments. Improvement was monitored via three video-based ratings (study start, end of the study period, and follow-up) using the Gastrointestinal Endoscopy Competency Assessment Tool (GiECAT). Outcomes were analysed using two-way analysis of variance with repeated measurements. Results: This study recruited 21 participants (12, intervention cohort; nine, control cohort); 58 videos were recorded. The intraclass correlation coefficient was 0.88 (95% CI 0.61-0.98; p < 0.001). The global rating scale (GRS) and checklist (CL) in GiECAT were analysed separately. No statistically significant main effects of cohort (GRS: F(1,16) = 2.84, p = 0.11; CL: F(1,16) = 1.06, p = 0.32), test (GRS: F(2,32) = 2.56, p = 0.09; CL: F(2,32) = 0.76, p = 0.48), or interactions between cohort and test were observed (GRS: F(2,32) = 1.16, p = 0.33; CL: F(2,32) = 1.01, p = 0.37). Conclusions: SRL in patient-based colonoscopy is feasible; however, no clear effect on performance scores was observed.

Dual purposes by design: exploring alignment between residents' and academic advisors' documents in a longitudinal program.

Longitudinal academic advising (AA) and coaching programs are increasingly implemented in competency based medical education (CBME) to help residents reflect and act on the voluminous assessment data they receive. Documents created by residents for purposes of reflection are often used for a second, summative purpose-to help competence committees make decisions-which may be problematic. Using inductive, thematic analysis we analyzed written comments generated by 21 resident-AA dyads in one large internal medicine program who met over a 2 year period to determine what residents write when asked to reflect, how this aligns with what the AAs report, and what changes occur over time (total 109 resident self-reflections and 105 AA reports). Residents commented more on their developing autonomy, progress and improvement than AAs, who commented far more on performance measures. Over time, residents' writing shifted away from intrinsic roles, patient care and improvement towards what AAs focused on, including getting EPAs (entrustable professional activities), studying and exams. For EPAs, the emphasis was on getting sufficient numbers rather than reflecting on what residents were learning. Our findings challenge the practice of dual-purposing documents, by questioning the blurring of formative and summative intent, the structure of forms and their multiple conflicting purposes, and assumptions about the advising relationship over time. Our study suggests a need to re-evaluate how reflective documents are used in CBME programs. Further research should explore whether and how documentation can best be used to support resident growth and development.

Feedback that Lands: Exploring How Residents Receive and Judge Feedback During Entrustable Professional Activities.

Introduction: Receiving feedback from different types of assessors (e.g., senior residents, staff supervisors) may impact trainees' perceptions of the quantity and quality of data during entrustable professional activity (EPA) assessments. We evaluated the quality of EPA feedback provided by different assessors (senior residents, chief medical residents/subspecialty residents, and staff) and explored residents' judgements of the value of this feedback. Methods: From a database of 2228 EPAs, we calculated the frequency of contribution from three assessor groups. We appraised the quality of 60 procedure-related EPAs completed between July 2019 and March 2020 using a modified Completed Clinical Evaluation Report Rating (CCERR) tool. Next, we asked 15 internal medicine residents to sort randomly selected EPAs according to their judgements of value, as an elicitation exercise before a semi-structured interview. Interviews explored participants' perceptions of quality of written feedback and helpful assessors. Results: Residents completed over 60% of EPA assessments. We found no difference in modified-CCERR scores between the three groups. When judging EPA feedback value, residents described a process of weighted deliberation, considering perceived assessor characteristics (e.g., credibility, experience with EPA system), actionable written comments, and their own self-assessment. Discussion: Like other recent studies, we found that residents contributed most to procedure-related EPA assessments. To the established list of factors influencing residents' judgements of feedback value, we add assessors' adherence to, and their shared experiences of being assessed within, EPA assessment systems. We focus on the implications for how assessors and leaders can build credibility in themselves and in the practices of EPA assessments.

Effectiveness of Life Goal Framing to Motivate Medical Students During Online Learning: A Randomized Controlled Trial.

Introduction: Educators need design strategies to support medical students' motivation in online environments. Prompting students to frame a learning activity as preparing them to attain their life goals (e.g., helping others) via their clinical practice, a strategy called 'life goal framing', may enhance their autonomous motivation, learning strategy use, and knowledge retention. However, for students with low perceived competence for learning (PCL), life goal framing may have an adverse effect. A randomized controlled trial was conducted to test the effectiveness of life goal framing and the moderating effect of students' PCL. Methods: First- and second-year medical students across four Canadian universities (n = 128) were randomized to receive a version of an online module with an embedded prompt for life goal framing, or one without. Students' motivation, learning strategy use, and knowledge retention were assessed. Differences between conditions on each outcome were estimated using Bayesian regression. Results: Students' PCL was a moderator for autonomous motivation but no other outcomes. The prompt did not have a statistically significant effect on any outcome, even for learners with high PCL, except for a small effect on link-clicking behaviour. Discussion: The results of this study suggest that learners' autonomous motivation is influenced by how they make meaning of instruction in terms of their future life goals and their present confidence. We cannot recommend life goal framing as an effective design strategy at this point, but we point to future work to increase the benefit of life goal framing for learners with high confidence.

Exploring mentorship in surgery: An interview study on how people stick together.

OBJECTIVE: The objective is to explore the processes contributing to how and why mentors and mentees initiate, maintain and grow in their mentorship relationships in surgery. BACKGROUND: To explore the processes contributing to how and why mentors and mentees initiate, maintain and grow their mentorship relationships in surgery. Evidence suggests that mentorship has a positive impact on physicians' success. Consequently, mentorship programmes have been incorporated into many medicine environments, albeit with variable success. METHODS: We designed an interview-based study using a constructivist grounded theory approach to explore the dynamics of mentorship between junior and experienced surgeons. Recruited mentees were asked to nominate a senior surgeon they identified as a mentor. Both mentee and mentors were then interviewed separately. Transcripts were analysed using constant comparison to a create a final coding framework and to generate themes. RESULTS: We interviewed nine faculty mentors and 10 junior faculty mentees. Our analysis identified key themes describing how to initiate, maintain and grow a mentorship relationship. Mentorship starts with ensuring a 'good fit', persists through satisfying a reciprocal loop with timely communication and deepens the relationship through cycles of mutual investment, learning, and success. Participants also discussed how to navigate through tensions to avoid relationship breakdown, balancing formality and friendship, knowing when to transition a relationship to a new dynamic and finding areas of realistic contribution. CONCLUSIONS: We found that successful mentorship relationships are viewed as dynamic and thus require active investment and shared responsibility between mentees and mentors. Our results also emphasise the value of co-regulation in the relationship, where cycles of mutual investment can contribute to mutual learning and growth.

Relinquishing control? Supervisor co-regulation may disrupt students' self-regulated learning during simulation-based training.

When uncertain, medical trainees often seek to co-regulate their learning with supervisors and peers. Evidence suggests they may enact self-regulated learning (SRL) strategies differently when engaged in self- versus co-regulated learning (Co-RL). We compared the impacts of SRL and Co-RL on trainees' acquisition, retention, and preparation for future learning (PFL) of cardiac auscultation skills during simulation-based training. In our two-arm, prospective, non-inferiority trial, we randomly assigned first- and second-year medical students to the SRL (N = 16) or Co-RL conditions (N = 16). Across two learning sessions separated by two-weeks, participants practiced and were assessed in diagnosing simulated cardiac murmurs. We examined diagnostic accuracy and learning trace data across sessions, and conducted semi-structured interviews to explore participants' understandings of their underlying choices and learning strategies. SRL participants' outcomes were non-inferior to Co-RL participants on the immediate post-test and retention test, but not on the PFL assessment (i.e., inconclusive). Analyzing interview transcripts (N = 31) generated three themes: perceived utility of initial learning supports for future learning; SRL strategies and sequencing of murmurs; and perceived control over learning across sessions. Co-RL participants regularly described relinquishing control of learning to supervisors and regaining it when on their own. For some trainees, Co-RL seemed to interfere with their situated and future SRL. We posit that transient clinical training sessions, typical in simulation-based and workplace-based settings, may not allow the ideal processes of Co-RL to unfold between supervisor and trainee. Future research must examine how supervisors and trainees can share accountability to develop the shared mental models that underlie effective Co-RL.

The fundamentals of Artificial Intelligence in medical education research: AMEE Guide No. 156.

The use of Artificial Intelligence (AI) in medical education has the potential to facilitate complicated tasks and improve efficiency. For example, AI could help automate assessment of written responses, or provide feedback on medical image interpretations with excellent reliability. While applications of AI in learning, instruction, and assessment are growing, further exploration is still required. There exist few conceptual or methodological guides for medical educators wishing to evaluate or engage in AI research. In this guide, we aim to: 1) describe practical considerations involved in reading and conducting studies in medical education using AI, 2) define basic terminology and 3) identify which medical education problems and data are ideally-suited for using AI.

Motivational Design for Web-Based Instruction in Health Professions Education: Protocol for a Systematic Review and Directed Content Analysis.

BACKGROUND: Web-based instruction plays an essential role in health professions education (HPE) by facilitating learners' interactions with educational content, teachers, peers, and patients when they would not be feasible in person. Within the unsupervised settings where web-based instruction is often delivered, learners must effectively self-regulate their learning to be successful. Effective self-regulation places heavy demands on learners' motivation, so effective web-based instruction must be designed to instigate and maintain learners' motivation to learn. Models of motivational design integrate theories of motivation with design strategies intended to create the conditions for motivated engagement. Teachers can use such models to develop their procedural and conceptual knowledge in ways that help them design motivating instruction in messy real-world contexts. Studies such as randomized controlled trials (RCTs) and other quasi-experimental designs that compare different motivational design strategies play a critical role in advancing models of motivational design. Synthesizing the evidence from those studies can identify effective strategies and help teachers and researchers understand the mechanisms governing why strategies work, for whom, and under what circumstances. OBJECTIVE: The planned review aims to analyze how studies comparing motivational design strategies for web-based instruction in HPE support and advance models of motivational design by (1) controlling for established risks to internal validity, (2) leveraging authentic educational contexts to afford ecological validity, (3) drawing on established theories of motivation, (4) investigating a wide breadth of motivational constructs, and (5) analyzing mediators and moderators of strategy effects. METHODS: The planned review will use database searching, registry searching, and hand searching to identify studies comparing motivational design strategies for web-based instruction, delivered to learners in HPE. Studies will be considered from 1990 onward. Two team members will independently screen studies and extract data from the included studies. During extraction, we will record information on the design characteristics of the studies, the theories of motivation they are informed by, the motivational constructs they target, and the mediators and moderators they consider. RESULTS: We have executed our database and registry searches and have begun screening titles and abstracts. CONCLUSIONS: By appraising the characteristics of studies that have focused on the motivational design of web-based instruction in HPE, the planned review will produce recommendations that will ensure impactful programs of future research in this crucial educational space. TRIAL REGISTRATION: PROSPERO CRD42022359521; https://tinyurl.com/57chuzf6. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/42681.

Re-examining the integration of routine and adaptive expertise: there is no such thing as routine from a motor control perspective.

The study of adaptive expertise in health professions education has focused almost exclusively on cognitive skills, largely ignoring the processes of adaptation in the performance of precision technical skills. We present a focused review of literature to argue that repetitive practice is much less repetitive than often perceived. Our main thesis is that all skilled movement reflects components of adaptive expertise. Through an overview of perspectives from the field of motor control and learning, we emphasize the interplay between the inherent noisiness of the human motor architecture and the stability of motor skill performances. Ultimately, we challenge the very idea of routine. Our goal is threefold: to reconcile common misconceptions about the rote nature of routine precision skill performance, to offer educators principles to enhance adaptive expertise as an outcome of precision skill training, and to expand the conversation between 'routine' and 'adaptive' forms of expertise in health professions education.

On embedding assessments of self-regulated learning into licensure activities in the health professions: a call to action.

How well have healthcare professionals and trainees been prepared for the inevitable demands for new learning that will arise in their future? Given the rapidity with which 'core healthcare knowledge' changes, medical educators have a responsibility to audit whether trainees have developed the capacity to effectively self-regulate their learning. Trainees who engage in effective self-regulated learning (SRL) skillfully monitor and control their cognition, motivation, behaviour, and environment to adaptively meet demands for new learning. However, medical curricula rarely assess trainees' capacity to engage in these strategic processes. In this position paper, we argue for a paradigm shift toward assessing SRL more deliberately in undergraduate and postgraduate programs, as well as in associated licensing activities. Specifically, we explore evidence supporting an innovative blend of principles from the science on SRL, and on preparation for future learning (PFL) assessments. We propose recommendations for how program designers, curriculum developers, and assessment leads in undergraduate and postgraduate training programs, and in licensing bodies can work together to develop integrated assessments that measure how and how well trainees engage in SRL. Claims about lifelong learning in health professions education have gone unmatched by responsive curricular changes for far too long. Further neglecting these important competencies represents a disservice to medical trainees and a potential risk to the future patients they will care for.

Dans quelle mesure les professionnels de la santé et les étudiants ont-ils été préparés aux exigences inévitables de nouveaux apprentissages qui se présenteront à eux à l'avenir? Étant donné la rapidité avec laquelle les «connaissances de base en matière de soins de santé¼ évoluent, les enseignants en médecine ont la responsabilité de vérifier si les étudiants ont développé la capacité d'autoréguler adéquatement leurs apprentissages. Ceux qui pratiquent efficacement l'apprentissage autorégulé (AAR) surveillent et contrôlent habilement leur cognition, leur motivation, leur comportement et leur environnement pour s'adapter à la nécessité de nouveaux apprentissages. Cependant, les programmes d'études médicales évaluent rarement la capacité des étudiants à s'engager dans ce processus stratégique. Dans cet exposé de position, nous plaidons en faveur d'un changement de paradigme vers une évaluation plus ciblée de l'AAR dans les formations doctorale et postdoctorale, ainsi que pour les activités d'évaluation. Plus précisément, nous explorons les résultats convaincants de l'emploi d'un mélange innovant de principes issus de la recherche en matière d'AAR et d'évaluations de la préparation à l'apprentissage futur. Nous proposons des recommandations pour une collaboration entre les responsables de la conception de programmes d'études, ceux de l'élaboration du cursus, ceux chargés de l'évaluation dans les programmes d'études prédoctorales et postdoctorales et les organismes responsables de l'octroi d'un titre de compétence en vue de créer des évaluations intégrées qui mesurent la méthode et la qualité de l'AAR chez les étudiants. Les programmes d'études tardent encore à traduire dans la pratique la reconnaissance de l'importance de l'apprentissage tout au long de la vie dans l'éducation médicale. Continuer à négliger ces compétences importantes ne ferait que nuire aux étudiants en médecine et potentiellement à leurs futurs patients.

Evaluation of an advanced critical care echocardiography program: a mixed methods study.

PURPOSE: Limited data exist on advanced critical care echocardiography (CCE) training programs for intensivists. We sought to describe a longitudinal echocardiography program and investigate the effect of distributed conditional supervision vs predefined en-bloc supervision, as well as the effect of an optional echocardiography laboratory rotation, on learners' engagement. METHODS: In this mixed methods study, we enrolled critical care fellows and faculty from five University of Toronto-affiliated intensive care units (ICU) between July 2015 and July 2018 in an advanced training program, comprising theoretical lectures and practical sessions. After the first year, the program was modified with changes to supervision model and inclusion of a rotation in the echo laboratory. We conducted semistructured interviews and investigated the effects of curricular changes on progress toward portfolio completion (150 transthoracic echocardiograms) using a Bayesian framework. RESULTS: Sixty-five learners were enrolled and 18 were interviewed. Four (9%) learners completed the portfolio. Learners reported lack of time and supervision, and skill complexity as the main barriers to practicing independently. Conditional supervision was associated with a higher rate of submitting unsupervised echocardiograms than unconditional supervision (rate ratio, 1.11, 95% credible interval, 1.08 to 1.14). After rotation in the echocardiography laboratory, submission of unsupervised echocardiograms decreased. CONCLUSION: Trainees perceived lack of time and limited access to supervision as major barriers to course completion. Nevertheless, successful portfolio completion was related to factors other than protected time in the echocardiography laboratory or unconditional direct supervision in ICU. Further research is needed to better understand the factors promoting success of CCE training programs.

RéSUMé: OBJECTIF: Il n'existe que peu de données sur les programmes de formation avancés en échocardiographie pour les soins intensifs (écho-USI) destinés aux intensivistes. Nous avons cherché à décrire un programme longitudinal d'échocardiographie et à étudier l'effet d'une supervision conditionnelle distribuée vs une supervision prédéfinie en bloc, ainsi que l'effet d'une rotation facultative en laboratoire d'échocardiographie, sur le niveau d'implication des apprenants. MéTHODE: Dans cette étude à méthodes mixtes, nous avons recruté des fellows en soins intensifs et des professeurs de cinq unités de soins intensifs (USI) affiliées à l'Université de Toronto entre juillet 2015 et juillet 2018 pour participer à un programme de formation avancée comprenant des conférences théoriques et des séances pratiques. Après la première année, le programme a été modifié en apportant des changements au modèle de supervision et en incluant une rotation dans le laboratoire d'écho. Nous avons mené des entretiens semi-structurés et étudié les effets des changements du programme d'études sur les progrès vers la réussite de la formation (150 échocardiogrammes transthoraciques) en utilisant un cadre bayésien. RéSULTATS: Soixante-cinq apprenants étaient inscrits et 18 ont été interviewés. Quatre (9 %) apprenants ont complété la formation. Les apprenants ont signalé que le manque de temps et de supervision ainsi que la complexité des compétences constituaient les principaux obstacles à une pratique autonome. La supervision conditionnelle était associée à un taux plus élevé de soumission d'échocardiogrammes non supervisés que la supervision inconditionnelle (ratio de taux, 1,11, intervalle crédible à 95 %, 1,08 à 1,14). Après la rotation dans le laboratoire d'échocardiographie, la soumission d'échocardiogrammes non supervisés a diminué. CONCLUSION: Les stagiaires ont perçu le manque de temps et l'accès limité à la supervision comme des obstacles majeurs à la réussite de la formation. Néanmoins, l'achèvement du cours était lié à des facteurs autres que le temps protégé au laboratoire d'échocardiographie ou la supervision directe inconditionnelle aux soins intensifs. D'autres recherches sont nécessaires pour mieux comprendre les facteurs favorisant le succès des programmes de formation en écho-USI.

Mastery versus invention learning: impacts on future learning of simulated procedural skills.

BACKGROUND: Invention and mastery learning approaches differ in their foundational educational paradigms, proposed mechanisms of learning, and potential impacts on learning outcomes. They also differ in their resource requirements. We explored the relative effects of 'invent and problem-solve, followed by instruction' (PS-I) learning compared to mastery learning (i.e., standards-based training) on immediate post-test and Preparation for Future Learning (PFL) assessments. PFL assessments measure learners' capacity to use their existing knowledge and strategies to learn about and solve novel problems. METHODS: In this non-inferiority trial, pre-clerkship medical students were randomized to either PS-I, Mastery Learning (ML), or instruction then practice (CON) during simulation-based training of infant lumbar puncture (LP). After a 2-week delay, participants returned to learn and complete a PFL assessment of simulated Knee Arthrocentesis. Two independent raters assessed performances with a 5-point global rating scale. RESULTS: Based on our non-inferiority margin, analyses showed that for both the immediate post-test and the PFL assessment, the PS-I condition resulted in non-inferior outcomes relative to the ML condition. Results for the CON condition were mixed with respect to non-inferiority compared to either PS-I or ML. CONCLUSIONS: We suggest cautiously that the PS-I approach was not inferior to the ML approach, based on skill acquisition and PFL assessment outcomes. With ML anecdotally and empirically requiring more time, greater faculty involvement, and higher costs, our findings question the preference ML has received relative to other instructional designs, especially in the healthcare simulation community. We encourage researchers to study the educational and resource impacts of instructional designs using non-inferiority designs.

Variable or Fixed? Exploring Entrustment Decision Making in Workplace- and Simulation-Based Assessments.

PURPOSE: Many models of competency-based medical education (CBME) emphasize assessing entrustable professional activities (EPAs). Despite the centrality of EPAs, researchers have not compared rater entrustment decisions for the same EPA across workplace- and simulation-based assessments. This study aimed to explore rater entrustment decision making across these 2 assessment settings. METHOD: An interview-based study using a constructivist grounded theory approach was conducted. Gastroenterology faculty at the University of Toronto and the University of Calgary completed EPA assessments of trainees' endoscopic polypectomy performance in both workplace and simulation settings between November 2019 and January 2021. After each assessment, raters were interviewed to explore how and why they made entrustment decisions within and across settings. Transcribed interview data were coded iteratively using constant comparison to generate themes. RESULTS: Analysis of 20 interviews with 10 raters found that participants (1) held multiple meanings of entrustment and expressed variability in how they justified their entrustment decisions and scoring, (2) held personal caveats for making entrustment decisions "comfortably" (i.e., authenticity, task-related variability, opportunity to assess trainee responses to adverse events, and the opportunity to observe multiple performances over time), (3) experienced cognitive tensions between formative and summative purposes when assessing EPAs, and (4) experienced relative freedom when using simulation to formatively assess EPAs but constraint when using only simulation-based assessments for entrustment decision making. CONCLUSIONS: Participants spoke about and defined entrustment variably, which appeared to produce variability in how they judged entrustment across participants and within and across assessment settings. These rater idiosyncrasies suggest that programs implementing CBME must consider how such variability affects the aggregation of EPA assessments, especially those collected in different settings. Program leaders might also consider how to fulfill raters' criteria for comfortably making entrustment decisions by ensuring clear definitions and purposes when designing and integrating workplace- and simulation-based assessments.

Applying activity theory to undergraduate medical curriculum reform: Lessons in contradictions from multiple stakeholders' perspectives.

PURPOSE: Medical school curricula require regular updating. We adopted an activity theory lens to conduct a holistic, multiple stakeholder-informed analysis of curricular reform, aiming to understand how the social relations between groups contribute to unanticipated tensions and outcomes. METHODS: A research assistant conducted semi-structured interviews with purposively sampled (N = 19) administrative staff, faculty course leads, faculty tutors, curriculum developers, change leaders and student leaders. The team applied a framework analysis to guide within and between stakeholder comparisons. RESULTS: Participants reported unique (N = 21) and cross-cutting (N = 17) contradictions underscoring emerging drivers of current and potential change. Unique contradictions raised by 1-2 groups represented seeds of change that had the potential to spread across all groups. By contrast, two general types of cross-cutting contradictions arose when one group had a dominant, confirming voice or two or more groups had contrasting perspectives. CONCLUSIONS: While finding contradictions was expected, our analysis profiled their nature and some of the specific tensions they raised across and within stakeholder groups. The activity theory lens provided an accessible way to unravel curricular reform into manageable units of analysis. Systematically identifying contradictions arising from curricular reform will help stakeholders collaborate with a shared purpose toward positive, sustained change.

Toward 'seeing' critically: a Bayesian analysis of the impacts of a critical pedagogy.

Critical reflection supports enactment of the social roles of care, like collaboration and advocacy. We require evidence that links critical teaching approaches to future critically reflective practice. We thus asked: does a theory-informed approach to teaching critical reflection influence what learners talk about (i.e. topics of discussion) and how they talk (i.e. whether they talk in critically reflective ways) during subsequent learning experiences? Pre-clinical students (n = 75) were randomized into control and intervention conditions (8 groups each, of up to 5 interprofessional students). Participants completed an online Social Determinants of Health (SDoH) module, followed by either: a SDoH discussion (control) or critically reflective dialogue (intervention). Participants then experienced a common learning session (homecare curriculum and debrief) as outcome assessment, and another similar session one-week later. Blinded coders coded transcripts for what (topics) was said and how (critically reflective or not). We constructed Bayesian regression models for the probability of meaning units (unique utterances) being coded as particular what codes and as critically reflective or not (how). Groups exposed to the intervention were more likely, in a subsequent learning experience, to talk in a critically reflective manner (how) (0.096 [0.04, 0.15]) about similar content (no meaningful differences in what was said). This difference waned at one-week follow up. We showed experimentally that a particular critical pedagogical approach can make learners' subsequent talk, ways of seeing, more critically reflective even when talking about similar topics. This study offers the field important new options for studying historically challenging-to-evaluate impacts and supports theoretical assertions about the potential of critical pedagogies.

How Clinical Supervisors Conceptualize Procedural Entrustment: An Interview-Based Study of Entrustment Decision Making in Endoscopic Training.

PURPOSE: Entrustment is central to assessment in competency-based medical education (CBME). To date, little research has addressed how clinical supervisors conceptualize entrustment, including factors they consider in making entrustment decisions. The aim of this study was to characterize supervisors' decision making related to procedural entrustment, using gastrointestinal endoscopy as a test case. METHOD: Using methods from constructivist grounded theory, the authors interviewed 29 endoscopy supervisors in the United States and Canada across multiple specialties (adult and pediatric gastroenterology, surgery, and family medicine). Semistructured interviews, conducted between April and November 2019, focused on how supervisors conceptualize procedural entrustment, how they make entrustment decisions, and what factors they consider. Transcripts were analyzed using constant comparison to generate an explanatory framework and themes. RESULTS: Three themes were identified from the analysis of interview transcripts: (1) entrustment occurs in varying degrees and fluctuates over time; (2) entrustment decisions can transfer within and across procedural and nonprocedural contexts; (3a) persistent static factors (e.g., supervisor competence, institutional culture, legal considerations) influence entrustment decisions, as do (3b) fluctuating, situated dynamic factors (e.g., trainee skills, patient acuity, time constraints), which tend to change from one training encounter to the next. CONCLUSIONS: In the process of making procedural entrustment decisions, clinical supervisors appear to synthesize multiple dynamic factors against a background of static factors, culminating in a decision of whether to entrust. Entrustment decisions appear to fluctuate over time, and assessors may transfer decisions about specific trainees across settings. Understanding which factors supervisors perceive as influencing their decision making has the potential to inform faculty development, as well as competency committees seeking to aggregate faculty judgments about trainee unsupervised practice. Those leading CBME programs may wish to invest in optimizing the observed static factors, such that these foundational factors are tuned to facilitate trainee learning and achievement of entrustment.

Making Concepts Material: A Randomized Trial Exploring Simulation as a Medium to Enhance Cognitive Integration and Transfer of Learning.

BACKGROUND: Simulation affords opportunities to represent functional relationships between conceptual (eg, anatomy) and procedural knowledge (eg, needle insertion technique) in ways that make them accessible to our many senses. Despite deprioritizing realism, such simulations may encourage trainees to create cognitive connections between these knowledge (ie, cognitive integration), which may improve transfer of learning. However, the impact of such "integrated instruction" has not been examined in simulation-based training. We developed integrated video- and simulator-based instructional modules for lumbar puncture training and compared their impacts on participants' retention, transfer, and conceptual knowledge. METHODS: During 1 hour of simulation-based training, we randomized 66 medical students to receive either (a) video-based procedural-only instruction, (b) integrated video-based instruction, or (c) integrated simulator-based instruction. One week later, we tested participants' retention and transfer performances and their conceptual knowledge on a written test. RESULTS: Simple mediation analyses revealed that compared with participants receiving procedural-only instruction, participants receiving integrated instruction had superior retention and transfer outcomes, mediated by gains in conceptual knowledge (all P < 0.01). We found no significant differences between the integrated groups for retention, transfer, or conceptual knowledge (all P > 0.01). CONCLUSIONS: We extended previous findings, showing integrated instruction (video- or simulator-based) improved trainees' conceptual knowledge, which mediated their improved retention and transfer. As an innovation, we demonstrated how simulators can facilitate cognitive integration by making abstract conceptual-procedural relationships material. In suggesting how researchers might capitalize further on simulator-based integration, we offer an alternative framework for designing simulations that emphasizes cognitive processes rather than simulator fidelity.

Getting everyone to the table: exploring everyday and everynight work to consider 'latent social threats' through interprofessional tabletop simulation.

In this methodological intersection article, we describe how we developed a new variation of the established tabletop simulation modality, inspired by institutional ethnography (IE)-informed principles. We aimed to design and conduct pilot implementations of this innovative tabletop simulation modality, which focused uniquely on everyday and everynight work, along with the factors that govern that work. In so doing, we aimed to develop a modality and preliminary findings that researchers and educators can use to simulate healthcare practices across longer episodes of care (i.e., time scales of hours or an entire day) and to detect the 'latent social threats' that can emerge during interprofessional clinical care.An interprofessional team designed tabletop simulation scenarios of interprofessional challenges during transfers of care on a labour and delivery (L&D) unit. Within each scenario, participants provided real-time explanations for their work and associated drivers, both independently and as a team. Thus, we combined 'think-aloud' and simulation principles to design tabletop simulation scenarios to elicit healthcare professionals' descriptions of how they collaborate in their work on the L&D unit. We completed a total of five tabletop simulations with eight participants (obstetricians, N = 2; midwives, N = 2; nurses, N = 5).The conversations stimulated by the tabletop simulation scenarios and debriefs allowed us to generate a preliminary understanding of the texts that govern and organize clinicians' everyday work processes. We generated data about longitudinal, multi-hour work processes in a condensed timeline, with opportunities to pause and probe, and with reduced focus on individual practitioner's competence.We believe our innovative tabletop simulation approach allowed us to examine clinical work in ways no other simulation permits. Participants described how the scenarios opened a productive dialogue between professional groups and suggested this simulation-based approach might contribute to enhanced interprofessional understanding and cultural change. We suggest that others can adapt our low-resource approach to understand clinicians' everyday work and to map how this work is governed by documents, like policies, with the end goal of facilitating system change and managing latent social threats.

Implicit and inferred: on the philosophical positions informing assessment science.

Assessment practices have been increasingly informed by a range of philosophical positions. While generally beneficial, the addition of options can lead to misalignment in the philosophical assumptions associated with different features of assessment (e.g., the nature of constructs and competence, ways of assessing, validation approaches). Such incompatibility can threaten the quality and defensibility of researchers' claims, especially when left implicit. We investigated how authors state and use their philosophical positions when designing and reporting on performance-based assessments (PBA) of intrinsic roles, as well as the (in)compatibility of assumptions across assessment features. Using a representative sample of studies examining PBA of intrinsic roles, we used qualitative content analysis to extract data on how authors enacted their philosophical positions across three key assessment features: (1) construct conceptualizations, (2) assessment activities, and (3) validation methods. We also examined patterns in philosophical positioning across features and studies. In reviewing 32 papers from established peer-reviewed journals, we found (a) authors rarely reported their philosophical positions, meaning underlying assumptions could only be inferred; (b) authors approached features of assessment in variable ways that could be informed by or associated with different philosophical assumptions; (c) we experienced uncertainty in determining (in)compatibility of philosophical assumptions across features. Authors' philosophical positions were often vague or absent in the selected contemporary assessment literature. Leaving such details implicit may lead to misinterpretation by knowledge users wishing to implement, build on, or evaluate the work. As such, assessing claims, quality and defensibility, may increasingly depend more on who is interpreting, rather than what is being interpreted.